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EFFICIENCY  OF  CLASSIFICATION: 

A  REVISION  OF  THE  BROGDEN  TABLE 


L  INTRODUCTION 

Classification,  and  associated  measures  of  effectiveness,  are  of  especial  interest  to 
the  Military  Services  because  of  the  nature  of  large  military  personnel  systems  (Johnson 
&  Ziedner,  1990;  Campbell  &  Russell,  1994;  Alley  Darby,  1994;  and  Rosse,  Whetzel, 
&  Peterson,  1994).  Large  numbers  of  applicants  and  available  military  jobs  motivate  the 
need  to  investigate  the  properties  of  the  assignment  of  groups  of  applicants  to  sets  of  jobs. 
Classification  efficiency  as  defined  by  Brogden  (1959)  refers  to  the  average  level  of 
productivity  when  a  group  of  applicants  are  assigned  to  a  set  of  jobs  using  full  least 
squares  predicted  performance  scores  as  the  basis  for  making  the  assignments.  Brogden 
proposed  the  mean  predicted  performance  (MPP)  as  the  index  of  classification  efficiency. 
MPP  is  the  mean  of  the  performance  scores  of  a  group  of  applicants  assigned  to  a  set  of 
jobs  when  that  assignment  has  been  made  with  a  methodology  which  accomplishes  an 
optimal  match  of  people  to  jobs.  Brogden’s  optimal  assignment  strategy  consisted  of 
computing  each  applicant’s  predicted  performance  scores  for  each  of  the  jobs  in  a  set  and 
then  assigning  an  individual  to  the  job  for  which  he  had  the  highest  performance  score. 
Brogden’s  work  with  the  largest  predicted  performance  score  was  based  on  earlier  results 
from  Tippett  (1925).  Tippett  was  concerned  with  the  statistical  issue  of  the  sampling 
distribution  of  extreme  values  of  scores  for  random  samples  of  varying  size  and  the 
implications  for  the  properties  of  range  statistics.  In  order  to  apply  Tippett’s  findings  to 
the  problem  of  assessing  classification  efficiency,  Brogden  made  the  following 
simplifying  assumptions:  1)  all  assignments  of  people  to  jobs  are  made  according  to  the 
highest  performance  score,  2)  there  is  an  infinite  supply  of  applicants,  and  3)  all  jobs  have 
equal  quotas  and  are  equally  important.  Under  these  three  reasonable  simplification 
assumptions  enabling  theoretical  derivation,  MPP  values  vary  with  the  applicant  rejection 
rate  and  the  validity  and  intercorrelation  of  the  performance  scores.  Brogden  calculated  a 
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table  (referred  to  as  the  Brogden  table  hereafter)  showing  the  theoretical  MPP  values  in 
the  multiple  job  system  of  one  to  ten  jobs  for  perfectly  valid  performance  estimates  and 
zero  correlation  of  estimates  across  jobs.  For  the  more  general  case  of  non-zero  but 
equally  correlated  job  performance  scores,  Brogden  proposed  an  adjustment  formula. 
Using  Monte  Carlo  simulation  to  extend  the  Brogden  table  to  500  jobs,  Alley  and  Darby 
discovered  that  Brogden’ s  adjustment  greatly  under  estimates  the  actual  MPP  in  the  cases 
of  non-zero  rejection  rates  and  provided  a  remedy.  It  has  been  an  open  problem  to 
establish  a  sound  and  more  accurate  adjustment  formula  for  the  Brogden  table.  Taking  a 
distribution-theoretic  approach,  this  study  establishes  an  adjustment  formula  that  gives 
more  accurate  results  than  the  Alley-Darby  formula.  Mathematical  details  of  the 
development  of  the  new  adjustment  formula  are  relegated  to  the  Appendix.^ 

II.  METHODS 

The  MPP  calculation  problem  studied  by  Brogden  admits  a  natural  formulation  in 
terms  of  certain  probability  distributions.  Brogden’s  assumptions  enabling  the  theoretical 
calculation  are  equivalent  to  the  following  probabilistic  model.  Mathematical  details  are 
in  the  appendix. 

Suppose  there  are  m  jobs.  The  applicant  population  represented  by  the  predicted 
performance  scores  can  be  treated  as  an  w-component  random  vector  [C,,C2,”',C,„] , 
where  Cj  is  the  performance  score  of  a  (randomly  selected)  applicant  on  the  jth  job. 

In  the  basic  situation  of  unit  validity  (if=l)  and  zero  correlation  between  the 
performance  scores  (r=0),  each  Cj  is  assumed  to  follow  the  standard  normal  distribution, 
and  all  Cj ’s  are  independent.  Because  each  applicant  is  assigned  to  the  job  corresponding 
to  the  highest  score,  the  classification  efficiency  for  the  assignment  is  calculated  from  the 
maximum  component  in  the  vector,  i.e. 


^  With  numerous  typographical  errors,  Rosse,  Whetzel,  and  Peterson  (1994)  discuss  the  calculation  of  the 
theoretical  MPP  in  general  terms.  Details  of  a  sound  development  under  the  Brogden  assumptions  are 
given  in  the  Appendix.  A  correction  of  the  fundamental  equation  (4)  in  Rosse,  Whetzel,  and  Peterson  is 
available  from  the  current  authors. 
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C*  =  max(C,,C,,  --,C„,)  . 

In  statistical  terminology,  C  *  is  the  largest  order  statistic  out  of  the  m  random  variables. 
The  mean  predicted  performance  (MPP)  after  assignment,  when  the  rejection  rate  is 
simply  the  expected  value  (or  mean)  of  the  largest  order  statistic  C  * .  By  a  known  fact  of 
the  largest  order  statistic  (See  e.g.  Larson,  1982,  pp.  318-319)  and  the  mathematical 
definition  of  mean,  the  theoretical  MPP  is  given  by  the  following  integral. 

CO 

(1)  MPPq-  Ja:/m[0(x)]™ '(j)(jc)c&  , 


where (t)(x)  =  (271 )  exp 


is  the  standard  normal  probability  density  function 


(pdf),  and  0(x)  =  |<|)(0^^^  is  the  standard  normal  cumulative  distribution  function  (cdf). 

-00 

The  entries  on  the  second  column  of  Table  1  in  Brogden  and  Table  A1  in  Alley  and 
Darby  are  the  approximated  values  of  the  integral  (1)  for  different  numbers  of  jobs.  When 
the  rejection  rate  p>0.  the  theoretical  MPP  is  the  following  integral: 


(2) 


MPPp 


1 

-  fxmrOfx)]**  '(j)(x)(ix:, 

l-pf 


where  L  is  the  -th  percentile  (quantile)  of  the  standard  normal  distribution. 

In  the  more  general  situation  where  the  validity  0  <  i?  <  1 ,  and  the  performance 
scores  C,  ,C2  ,  are  equally  correlated  with  pairwise  correlation  0  <  r  <  1 ,  Brogden 
proposed  a  model  in  which  all  the  m  scores  are  related  to  a  common  variable  K,  plus  an 
independent  random  perturbation  following  the  normal  distribution.  Symbolically, 

Cj=K  +  Cj,  j  =  l,2,...,m. 


where  K  follows  the  normal  distribution  with  mean  zero  and  variance  ,  and 
C,  ,C'2  ,  are  independent  and  identically  distributed  by  the  normal  distribution 

with  mean  zero  and  variance  i?^(l-r),  and  are  independent  of  K.  This  model  does 
provide  the  desired  theoretical  distribution  of  the  performance  scores,  but  in  this 
situation,  a  simple  integral  representation  of  the  theoretical  MPP  like  equation  (1)  or  (2) 
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no  longer  exists.  For  practice,  a  manageable  adjustment  for  the  table  entries  is  desirable. 
The  adjustment  formula  given  by  Brogden  is  correct  only  when  the  rejection  rate  is  zero. 
Much  improvement  is  obtained  by  Alley  and  Darby  based  on  empirical  studies  using 
Monte  Carlo  simulation.  With  the  known  distributional  properties  of  the  performance 
scores  and  the  properties  of  the  quantile  function  (the  inverse  function  of  the  cdf),  an 
approximation  formula  can  be  established  for  adjusting  the  table  values  to  a  much  better 
accuracy.  This  formula  is  given  in  the  following  section  together  with  a  revised  Brogden 
table. 

With  present-day  computing  power,  the  integrals  (1)  and  (2),  together  with  the 
quantity  L  in  (2),  can  be  easily  calculated  with  numerical  routines.  The  integrals  can  be 
calculated  to  a  satisfactory  accuracy  with  a  trapezoidal  rule.  Functions  for  calculating  L 
are  readily  available  in  statistical  packages  such  as  SAS  and  S-plus. 

III.  RESULTS 

Table  1  is  a  revised  Brogden  table  of  theoretical  MPP  values  computed  using  S- 
plus.  The  revised  Brogden  table  expresses  MPP  in  standard  score  metric,  mean  of  zero 
and  standard  deviation  of  one,  as  does  the  original  table.  Note  that  a  new  column  V  is 
added.  In  the  revised  table  m  denotes  the  number  of  jobs  and  table  entries  are  the  mean 
performance  values  for  different  numbers  of  jobs  and  rejection  rates  when  the  validity 
R  =  \  and  the  between-job  correlation  r  =  0 .  The  quantity  V  is  used  for  adjustment 
when  R^  \  and/or  r  ^0 .  The  adjustment  formula  is 

R^M,+^4v\\-r)  +  r[M^  -  M,), 

where  is  the  first  column  table  entry  for  the  given  number  of  jobs  and  the  rejection 
rate  p  =  0  (pure  classification),  is  the  table  entry  for  the  given  number  of  jobs  and 
the  rejection  rate  p  (selection  and  classification).  For  example,  for  /?  =  1,  r  =  0.85,  16 
jobs,  and  rejection  rate  p  =  0.6,  the  adjusted  mean  performance  is 
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Vl- 0.85(1.766)  +  — 7(0-5432)- (1  -  0.85)  +  0.85(2.295  - 1.766)  =  1.604 . 

The  table  values,  other  than  V ,  are  as  in  the  original  Brogden  table  and  the  Alley 
and  Darby  extension.  Inspection  of  the  table  indicates  that  an  increase  in  jobs  produces 


Table  1 

Revised  Brogden  Table 


#  Jobs 


_  Rejection  Rate  (percentage) 

O _ 12 _ _ 22 _ 12 _ 52 _ 60  ”70  80  90 


1 

1.0000 

0.000 

0.195 

0.350 

0.497 

2 

0.8257 

0.564 

0.721 

0.847 

0.969 

3 

0.7480 

0.846 

0.986 

1.100 

1.210 

4 

0.7012 

1.029 

1.159 

1,265 

1.368 

5 

0.6690 

1.163 

1.285 

1.386 

1.485 

6 

0.6449 

1.267 

1.384 

1.481 

1.576 

7 

0.6260 

1.352 

1.465 

1.559 

1.651 

8 

0.6107 

1.424 

1.533 

1.625 

1.715 

9 

0.5978 

1.485 

1.592 

1.682 

1.769 

10 

0.5868 

1.539 

1.643 

1.731 

1.817 

11 

0.5773 

1.586 

1.689 

1.775 

1.860 

12 

0.5690 

1.629 

1.730 

1.815 

1.899 

13 

0.5614 

1.668 

1.767 

1.851 

1.933 

14 

0.5547 

1.703 

1.801 

1.884 

1.965 

15 

0.5487 

1.736 

1.832 

1.914 

1.995 

16 

0.5432 

1.766 

1.861 

1.942 

2.022 

17 

0.5381 

1.794 

1.888 

1.968 

2.047 

18 

0.5334 

1.820 

1.913 

1,993 

2.071 

19 

0.5291 

1.844 

1.937 

2.015 

2.093 

20 

0.5251 

1.867 

1.959 

2.037 

2.114 

100 

0.4294 

2.508 

2.580 

2.642 

2.705 

200 

0.4009 

2.746 

2.813 

2.871 

2.929 

300 

0.3865 

2.878 

2.941 

2.998 

3.054 

400 

0.3772 

2.968 

3.030 

3.085 

3.140 

500 

0.3704 

3.037 

3.097 

3.151 

3.205 

0.644 

0.798 

0.966 

1.159 

1.400 

1.755 

1.092 

1.221 

1.365 

1.531 

1.741 

2.057 

1.322 

1.441 

1.573 

1.727 

1.923 

2.220 

1.473 

1.586 

1.711 

1.858 

2.045 

2.330 

1.586 

1.693 

1.813 

1.955 

2.136 

2.413 

1.674 

1.778 

1.894 

2.032 

2.208 

2.479 

1.746 

1.848 

1.961 

2.096 

2.268 

2.534 

1.807 

1.907 

2.018 

2.149 

2.319 

2.580 

1.860 

1.957 

2.067 

2.196 

2.363 

2.621 

1.906 

2.002 

2.110 

2.237 

2.402 

2.657 

1.948 

2.042 

2.148 

2.274 

2.437 

2.689 

1.985 

2.078 

2.183 

2.307 

2.469 

2.718 

2.019 

2.111 

2.214 

2.338 

2.497 

2.745 

2.050 

2.141 

2.243 

2.365 

2.524 

2.769 

2.078 

2.168 

2.270 

2.391 

2.548 

2.792 

2.105 

2.194 

2.295 

2.415 

2.570 

2.813 

2.129 

2.218 

2.318 

2.437 

2.592 

2.832 

2.152 

2.240 

2.339 

2.458 

2.611 

2.850 

2.174 

2.261 

2.360 

2.477 

2.630 

2.868 

2.194 

2.281 

2.379 

2.495 

2.647 

2.884 

2.771 

2.843 

2.925 

3.024 

3,154 

3.360 

2.991 

3.058 

3.136 

3.229 

3.352 

3.547 

3.113 

3.179 

3.253 

3.344 

3.463 

3.653 

3.198 

3.261 

3.334 

3.423 

3.540 

3.727 

3.262 

3.324 

3.396 

3.483 

3.599 

3.783 

an  increase  in  expected  performance  with  R  and  r  held  constant  at  1  and  0  respectively. 
As  noted  in  Alley  and  Darby,  R  and  r  typically  change  as  the  number  of  jobs  increase. 
This  change  is  primarily  attributable  to  three  separate  effects:  (a)  the  increase  in 
opportunity  for  alternative  job  assignments,  (b)  a  potential  increase  in  validity  due  to 
additional  performance  prediction  composites,  and  (c)  a  potential  decrease  in  the  average 
among-job  intercorrelation,  if  the  additional  jobs  are  sufficiently  dissimilar  from  those 
already  defined. 
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The  performance  estimates  computed  with  the  revised  adjustment  procedure  are 
compared  with  empirical  estimates  of  performance  generated  from  simulation  results 
using  SAS  (1985)  reported  in  Alley  and  Darby.  In  addition  to  the  empirical  comparison, 
a  comparison  of  the  revised  adjustment  estimates  with  those  of  Brogden  and  Alley  and 
Darby  yields  the  results  in  Table  2.  First  note  that  for  multiple  jobs,  m>0,  and  the  pure 
classification  case,  /?  =  0 ,  the  second  term  of  the  revised  adjustment  formula  is  zero 


Table  2 

Comparison  of  Adjustment  Procedures  for  R=l. 


Condition 

Empirical 

Estimated 

#  Jobs 

Rejection 

Rate 

r 

Brogden  (%) 

Alley-Darby  (%) 

Revised  (%) 

2 

0.4 

.45 

1.00 

.81  (81%) 

.97  (97%) 

.999(100%) 

2 

0.4 

.85 

.85 

.42  (49%) 

.81  (95%) 

.842  (99%) 

2 

0.6 

.45 

1.28 

1.02  (80%) 

1.27  (99%) 

1.299(101%) 

2 

0.6 

.85 

1.16 

.53  (46%) 

1.12(97%) 

1.165  (100%) 

4 

0,4 

.45 

1.30 

1.10(84%) 

1.25  (97%) 

1.301  (100%) 

4 

0.4 

.85 

1.02 

.57  (56%) 

.97  (95%) 

1.001  (98%) 

4 

0.6 

.45 

1.58 

1.27  (80%) 

1.52  (96%) 

1.588  (100%) 

4 

0.6 

.85 

1.32 

.66  (50%) 

1.26  (95%) 

1.333  (101%) 

8 

0.4 

.45 

1.57 

1.34  (85%) 

1.51  (96%) 

1.564(100%) 

8 

0,4 

.85 

1.17 

.70  (60%) 

1.09  (93%) 

1.149  (98%) 

8 

0.6 

.45 

1.84 

1.50  (82%) 

1.76  (96%) 

1.843  (100%) 

8 

0.6 

.85 

1.48 

.79  (53%) 

1.38(93%) 

1.478(100%) 

16 

0.4 

.45 

1.82 

1.56  (86%) 

1.73  (95%) 

1.798  (99%) 

16 

0.4 

.85 

1.29 

.82  (63%) 

1.21  (93%) 

1.274  (99%) 

16 

0.6 

.45 

2.06 

1.71  (83%) 

1.96  (95%) 

2.071  (100%) 

16 

0.6 

.85 

1.59 

.89  (56%) 

1.49  (94%) 

1.604(101%) 

Note.  In  the  Empirical  Column  are  estimates  of  the  actual  MPP  obtained  by  simulation  in  Alley  &  Darby.  Adjustment  formulas 
should  produce  values  close  to  these  estimates 


because  =  M^;  the  first  term,  Ryj\~rMf^,  is  simply  Brogden’s  formula.  Moreover, 

this  expression  coincides  with  the  Alley  and  Darby  adjustment  given  in  different  notation. 
All  three  adjustment  formulas  agree  in  the  pure  classification  case.  For  multiple  jobs. 
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w>  0,  and  the  selection  and  classification  case,  /?>  0,  the  appropriate  adjustment  is  not 

simply  R^\-rMp  as  proposed  by  Brogden.  Alley  and  Darby  suggest,  for  a  given 

rejection  rate,  taking  the  difference  between  the  w  job  performance  estimate  (selection 
and  classification)  and  the  single  job  performance  estimate  (pure  selection),  applying  the 
Brogden  adjustment  to  that  difference,  and  then  adding  the  resulting  quantity  to  the  single 
performance  estimate.  The  revised  adjustment  formula  states  that  the  theoretically  proper 
adjustment  to  the  multiple  job  estimate  is  the  Brogden  adjustment  in  the  pure 
classification  case,  /?  =  0 ,  plus  the  properly  scaled  difference  between  the  m  job 

performance  estimate  (selection  and  classification)  and  the  pure  classification 
performance  estimate. 

Performance  estimates  in  Table  2  for  the  three  adjustment  formulas  are  in  the 
three  columns  under  the  label  ‘Estimated  M^\  In  these  three  columns  the  numbers  in 

parenthesis  are  the  percentage  of  the  empirical  value  accounted  for  by  the  difference 
between  the  empirical  value  and  the  revised  value.  Empirical  estimates  were  calculated 
for  combinations  of  the  following  conditions,  two  rejection  rates,  .4  and  .6;  four  different 
multiple  job  systems,  2,  4,  8,  and  10  jobs;  and  two  values  of  average  between  job 
correlation,  .45  and  .85.  Inspection  of  Table  2  reveals,  as  shown  in  Alley  and  Darby,  that 
Brogden’ s  adjustment  underestimates  mean  performance  by  varying  degrees  for  the 
multiple  job  system.  Estimates  computed  with  the  Alley  and  Darby  adjustment  formula 
are  much  closer  to,  but  slightly  underestimate,  the  empirical  values.  The  revised 
estimates  are  very  close  to  the  empirical  estimates  with  only  slight  variation  due  to 
sampling  error. 


IV.  CONCLUSION 

The  distributional  approach  to  revising  the  Brogden  table  reveals  some  insights 
into  the  theoretical  MPP  calculation,  and  consequently  a  much  more  accurate  adjustment 
formula  is  established.  It  is  possible  to  extend  this  approach  to  other  theoretical 
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measurement  calculations  in  personnel  and  education  psychology,  such  as  extending  the 
Taylor-Russell  tables  (Taylor  &  Russell,  1939;  Alley,  Darby,  &  Cheng,  1996)  to  the 
situation  of  multiple  equally  correlated  jobs.  Work  along  this  line  constitutes  further 
research  efforts. 

Caution  should  be  exercised  when  applying  the  values  in  Table  1.  Real  world 
personnel  systems  seldom  exhibit  ideal  characteristics  such  as  perfect  or  even  near  perfect 
validities  and  zero  or  near  zero  between  job  intercorrelations.  Table  2  gives  an  indication 
of  the  reduction  in  performance  that  can  occur  with  non-zero  between  job 
intercorrelations.  Applications  of  Table  1  should  be  made  with  a  conservative  eye  toward 
overstating  possible  gains  in  performance. 
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APPENDIX 

THE  DISTRIBUTION  THEORY  OF  THE  BROGDEN  TABLE 
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Preliminary 


Some  concepts  and  properties  of  probability  distributions  on  the  real  line,  especially  the  quantile 
functions,  are  given  here  for  future  references.  Let  Xbs  a  continuous  random  variable  on  the  real  line 
(-00,  oo) .  The  probability  distribution  of  X  is  determined  by  its  probability  density  function  (pdf) 

00 

/(x) ,  which  is  non-negative  and  j*  f  (t)dt  =  1 .  The  cumulative  distribution  function  (cdf)  is 


~  \f  ~  Pr(X  <  x)  =  {the  area  under  the  pdf  graph  left  to  x  }. 


Note  that  the  cdfF(x)  always  has  values  between  0  and  1,  and  is  non-decreasing.  For  the  current 
purpose  it  will  be  technically  simple  to  assume  that  F(x)  is  strictly  increasing.  The  quantile  junction  (qf) 
ofX,  denoted  by  ^ ,  is  the  inverse  function  of  the  cdf  F .  That  is,  for  any  0  <  t/  <  1 , 

Q(u)  =  (m)  =  {the  value  X  such  that  F(x)  =  u}  =  {the  value  X  such  that  the  area  under  the  pdf 

graph  left  to  x  is  «>.  Q(u)  is  called  the  a-th  quantile  of  the  random  variable  X .  Some  special  quantiles 
are:  the  1st  quartile  0(0.25) ,  the  median  (2nd  quartile)  0(0.5) ,  the  3rd  quartile  0(0.75) ,  the  40th 
percentile  0(0.4) ,  etc. 

Example  A.1  Suppose  that  X  follows  the  exponential  distribution  with  pdf  /  (x)  =  e”*,0<x<oo. 


X  X 

Then  the  cdf  is  F(x)  =  J/ (t)dt  =  | e~‘dt  =  1  -  e'* .  The  quantile  function  0(m)  is  obtained  by 

0  0 

setting  F(x)  =  u  and  solving  for  x.  We  have  F(x)  =  1  -  =  u,  thus  e"*  =  1  -  ir ;  taking  the 

natural  logarithm  on  both  side  gives  x  =  -  Iog(l  -  u) .  Hence  the  qf  0(m)  =  -  log(l  -  «),  0  <  M  <  1 . 
Example  A.2  Suppose  that  X  follows  the  standard  normal  distribution  iV^(0,l)  with  mean  0  and 


variance  1,  and  pdf  {^(x)  = 


The  qfis 

Q(u)  =  O  *  (u) .  The  cdf  (and  consequently  the  qf)  does  not  have  closed-form  expression  in  this  case. 
They  can  be  calculated  numerically  in  many  statistical  software  packages.  In  SAS,  the  cdf  O  is  the 

PROBNORM  function,  and  the  qf  <I>”*  is  the  PROBIT  function. 

The  quantile  function  has  many  useful  properties,  some  of  which  are  given  below. 

Proposition  A.1  The  cdf  and  the  qf  are  related  by  Q^Fixj)  =  X  for  any  X  inside  the  range  of  the 
random  variable  X ,  and  F{Q{u))  =  U  for  any  0  <  «  <  1 . 
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Proposition  A-2  As  u  approaches  to  0,  the  qf  Q{u)  approaches  to  the  smallest  possible  value  of  the 
random  variable  X ;  as  u  approaches  to  1,  Q(u)  approaches  to  the  largest  possible  value  of  X . 

These  two  properties  follow  directly  from  the  definitional  fact  that  the  qf  Q  is  the  inverse 
fimction  of  the  cdf  F . 


Proposition  A.3  The  expected  value  of  X ,  EJf,  is  equal  to  the  integral  E  X  j  Q(u)du. 

0 

Proof:  By  definition  of  expected  value  [Larson  1982,  p.ll8.  Definition  3.3.1  (b)], 

00  00 

EX=^:rfix)dx=  \xdF{x). 

-><»  -00 

Now  change  variable  by  setting  F(x)  =  u .  Then  by  Proposition  A.  I,  x  =  Q(u) ,  and  by  Proposition 
A.2,  the  range  of  u  is  the  unit  interval  (0, 1).  Hence  the  integral  becomes 

EX=]xdF(x)  =  \Q{u)du, 

—00  0 

completing  the  proof 

Example  A.3  Consider  the  exponential  random  variable  in  Example  A.I.  The  expected  value  is 
00  00  1  1 

EX=^xf{x)dx  =  ^xe~'‘dx  =  \.  Alternatively,  EX =^Q(u)du  =  f\og(y-u)du  =  \. 

0  0  0  0 

Proposition  A.4  Let  0  <  p  <\,  The  conditional  expected  value  of  X  given  that  X  >Q(j?)(i,e.  given 

1  ^ 

that  X  is  no  less  than  its  p-th  quantile),  is  given  by  - j"  Q(li)du. 

^  P  p 

The  proof  is  similar  to  that  of  Proposition  A.3;  it  is  omitted  to  keep  the  technicality  at  a  reasonable  level. 
Note  that  the  crucial  differences  here  from  the  unconditional  case  in  Proposition  A.3  are  the 
1/  (I-  p)  factor  and  the  lower  limit  of  integral  being  p,  not  0. 

Example  A.4  Consider  again  the  exponential  random  variable  in  Example  A.I.  Let  p  =  0.4 .  The  40th 
percentile  is  then  Q(0A)  =  -log(l  -  0.4)  =  -log(0.6)  .  The  conditional  expected  value  of  X  given 
that  X  is  greater  than  or  equal  to  its  40th  percentile  is  then 


— J Q{u)du  =  — ^  f-log(l  -  u)du  »  1.51 1. 
\-pi  I-0.4J 


p  -  -  ■  •  0.4 

For  the  standard  normal  variable  in  Example  A.1,  the  conditional  expected  value  is 


.  1 

0.644. 

l-04i4 
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Proposition  A-5  Let  X  be  a  random  variable  with  quantile  function  Qj^(u),  and  let  Y  =  A  +  BX , 
with  A  and  B  real  numbers.  Then  the  quantile  function  of  Y  is  Qy  (w)  =  A  +  BQ^  («) . 


The  Brogden  Table 

Brogden’s  (1959)  assumptions  are  equivalent  to  treating  the  applicants’  predicted  performance 
scores  as  certain  random  variables  and  the  applicant  population  being  represented  by  the  joint  probability 
distribution  of  the  performance  scores.  In  tune  with  Brogden’s  notation,  let  m  be  the  number  of  jobs,  and 
let  C,  jCjjA  ,C„  denote  the  predicted  performance  scores.  It  is  assumed  that  the  m  scores  are 

standardized  to  have  a  common  mean  0  and  a  common  variance  ,  which  is  the  validity.  It  is  further 
assumed  that  the  performance  scores  Cj ,  Cj ,  A  ,  C„  jointly  follow  an  m-variate  normal  distribution. 

Since  every  applicant  is  assigned  to  the  job  corresponding  to  the  highest  score,  the  allnratinn 
efficiency  is  simply  the  largest  of  ,C„  :  C*  =  max(C,,C2,A  ,C„)  ,  i.e.  the  largest  order 

statistic  of  the  m  random  variables. 

Brodgen  (1959)  considered  calculating  the  mean  allocation  efficiency  in  4  cases;  (1)  uncorrelated 
performance  scores  and  zero  rejection  rate,  (2)  uncorrelated  performance  scores  and  non-zero  rejection 

rate,  (3)  equally  correlated  performance  scores  (i.e.  all  (C(,Cy)  pairs  have  common  correlation 

0  <  r  <  1 )  and  zero  rejection  rate,  and  (4)  equally  conelated  performance  scores  and  non-zero  rejection 
rate.  Each  of  cases  is  discussed  below. 

Case  1:  imcorrelated  performance  scores  and  zero  rejection  rate  (r  =  0,  p  =  0).  In  this  case  the 
scores  C,  jCjjA  ,C,„  are  independent  and  identically  distributed  (i.i.d.)  as  normal  with  mean  0  and 

variance  R^  —  N^,  R}  ^ .  For  convenience  first  let  i?  =  1 .  Then  the  Cy  ’s  have  the  cortunon  standard 
normal  probability  density  function  (pdf)  =  (2tr)  expl'^  and  the  cumulative  distribution 

X 

fimction  (cdf)  ^(x)  =  J  •  The  mean  allocation  efficiency  (or  MPP)  is  nothing  but  the  expected 

-00 

value  of  the  maximum  score  C*  =  max(Cj ,  Cj  ,A  ,  C„)  .  Because  C*  is  the  largest  order  statistic  out 
of  the  m  scores,  by  a  known  fact  of  the  largest  order  statistic  (Larson  1982,  pp.  3 18-3 19)  and  the  normal 
distribution  assumption  of  the  performance  scores,  C*  has  cdf 

(3)  F(X)  =  [<D(X)]'",  -QO<X<QO, 

and  pdf 
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(4) 


-  00  <  X  <  00, 


/W  =  ^F(x)  =  <n[®wp'  «((x). 

Application  of  the  definition  of  expected  value  [Larson  1982,  p.  1 18,  Definition  3.3.1  (b)]  yields 

00  CO 

(5)  MPP^  =  \xf  (x>&  =  J  x»i[a)(x)]'^'  <^(x)dx  , 

—00  —00 

which  is  exactly  equation  (1).  Expected  values  and  standard  deviation  of  C  *  for  several  values  of  m  are 
calculated  in  Table  I,  Tippett  (1925). 

Alternatively,  the  quantile  fiinction  (qf)  Q(u)  of  C  *  can  be  obtained  by  setting  the  cdf  (3)  to 
M  and  solving  for  x:  F(x)  =  [O(x)]’"  =  M,  so  <I>(x)  =  and  x  =  Hence  theqf  of 

C  *  is 

(5a)  =  0<«<1. 

Then  by  Proposition  A.3,  the  alternative  expression  of  the  expected  value  of  C  *  (the  MPP)  is 

1 

(5b)  iWPPo  = 

0 

If  the  validity  R^\,  the  MPP  is  simply  Ri^MPP^  )  because  C  *  is  just  scaled  by  R . 

Case  2:  uncorrelated  performance  scores  and  non-zero  rejection  rate  (r  =  0,  p  >  0).  The 
distributions  of  the  performance  scores  and  the  maximum  score  C  *  remain  the  same  as  in  Case  1.  But  if 
the  rejection  rate  p>0,  i.e.  only  the  upper  (l  -  portion  of  the  applicant  population  is  selected,  then  the 

allocation  efficiency  is  the  conditional  expected  value  of  C  *  given  that  C  *  is  greater  than  or  equal  to 
its  pth  percentile  (quantile).  It  is  enough  to  only  recall  at  this  point  that  under  the  given  distributional 
assumptions,  the  pth  quantile  of  C  *  is  the  number Z  such  that  the  probability  of  C*^  L  is  p,  i.e. 

F{L)  =  p  with  F  the  cdf  of  C  *  given  in  equation  (3)  (i.e.  the  cumulative  area  under  the  pdf  graph  1^ 
toZ  is  p.)  Equivalently,  the  probability  of  C*>  L  is  (l  -  /?)  (i.e.  the  cumulative  area  under  the  pdf 
graph  right  toZ  isl  — /?.)  For  convenience,  again  first  let  /?  =  1.  Following  the  definition  of  a 
conditional  probability  law,  (Larson,  1982,  pp.  61-65,  241-243),  the  conditional  cdf  of  C*  given 
C*>L  is  (restricting  x'^L) 


FMoV  Pr(iSC*£A:)  F(x)  -  F(i)  _  -  P 

- - - w—  1-p  ■ 

and  F{x\p)  =  0  if  x  <  L.  The  conditional  pdf  of  C*  given  it  is  greater  than  or  equal  to  its  /»th 

percentile  is  then 
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(6)  /  (x\p)  =  -^F^xIp)  =  — /w[a)(x)]'^’  ^(x),  L^x  <00. 

ax  P 

The  MPP  corresponding  to  rejection  rate  p  is  the  conditional  expected  value 

00  oo 

(7a)  MPPj,  =  J  :rf{x\p)dx  =  (1  -  /?)“*  J  x»i[<I>(x)]""*  ^(x^abc , 

L  L 

which  is  exactly  equation  (2).  Note  that  in  the  integral  the  conditional  pdf  f(x\p),  not  the 
unconditional  one  in  (4),  is  used. 

By  Proposition  A.  4  and  the  qf  expression  of  C  *  in  (5a),  the  alternative  expression  of  the 
conditional  expected  value  (the  MPP)  is 

(7b)  = 

If 1  then  the  MPP  is  simply  Ri^MPP^  ) ,  for  the  same  reason  as  before. 

The  same  as  establishing  (5a),  because  F{L)  =  /7 ,  Z  can  be  obtained  by  setting  the  cdf  (3)  to  p 
and  solving  for  x,  giving  L  =  x  =  with  the  inverse  function  of  the  standard  normal 

cdf  (the  PROBIT  hmction  in  SAS).  It’s  interesting  to  note  that  the  pth  quantile  of  C*  ,  Z ,  is  the 
-th  quantile  of  the  normal  distribution. 

Case  3:  equally  correlated  performance  scores  (i.e.  all  pairs  have  common  correlation 

0  <  r  <  1 )  and  zero  rejection  rate.  For  this  more  general  case  Brogden  proposed  the  model  that  the  job 
performance  scores  are  related  to  a  common  latent  variable  K,  plus  independent  normal  random 

perturbation:  Cj  =  K  +  C' j ,  j  =  .  In  order  that  all  Cj ’s  follow  the  distribution 

and  are  equally  correlated,  it  is  assumed  that  AT  follows  the  N^O,  R^rj ,  and  C\,C2,A,C„  arei.i.d. 

N^,R^  (l  -  r)^and  are  independent  of  K.  This  model  then  provide  the  desired  theoretical  distribution 
of  the  performance  scores.  Now  the  allocation  efficiency 

(8)  C*  =  max(C,.C„A  ,C.)=  X+max(C’,  ,C,  ,A  ,C.)  =  AT  +  C*’, 

where C*'  =  max(C7j  ,C'2  ,A  Recall  that,  as  in  Case  1,  when  the  rejection  rate p  =  0,  the  MPP 

is  the  expected  value  (or  mean)  of  C  *  .  Because  the  mean  of  K  is  zero  and  mean  is  additive,  (8)  gives 
that  the  mean  of  C  *  (the  MPP)  is  simply  the  mean  of  K  plus  the  mean  of  C*' ,  which  is  equal  to  the 
expected  value  (or  mean)  of  C*' .  Furthermore,  because  of  the  distributional  assumption  on  the  C/s, 
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C*'  differs  from^  C  *  in  Case  1  only  by  a  scaling  factor  R-Jl-r  .  Therefore  the  MPP  in  this  case  is 
R-Jl  —  rMPP^ ,  the  Brogden  adjustment 

Case  4;  equally  correlated  performance  scores  and  non-zero  rejection  rate.  Brogden  attempts  to 
completely  reduce  the  correlated  job  score  cases  to  the  above  independent  job  score  cases  by  observing  that 
the  common  variable  K  has  zero  mean,  so  that  C  *  has  the  same  expected  value  as  C*' .  However,  this 
reduction  is  correct  only  when  the  rejection  rate  p=0.  The  reason  Brogden’s  adjustment  fails  in  the  p>0 
case  lies  in  the  fact  that  in  the  non-zero  rejection  cases  the  MPP  is  the  conditional  expected  value  of  C  * 
conditioned  on  its  being  greater  than  or  equal  to  its  pth  percentile  L ,  and  unfortunately  for  correlated  job 
scores,  L  is  no  longer  simply 

The  exact  probability  distribution  of  C  *  in  this  case  is  rather  complex.  Consequently,  simple 

integral  expressions  for  the  theoretical  MPP  like  those  in  equations  (1)  and  (2)  no  longer  exist  In  order  to 
establish  an  adjustment  formula  in  the  same  spirit  as  those  of  Brogden  and  Alley  &  Darby  (1994),  a 
distributional  approximation  method  is  employed.  Equation  (8)  shows  that  C*  is  C*'  plus  an 

independent  normal  random  variable  K  with  mean  zero  and  standard  deviation  i?-\/r<l.  And  as 

mentioned  before,  C*'  differs  from  the  C  *  in  Case  1  only  by  a  scaling  fector  R-Jl  —  r  ,  so  its  exact 
probability  distribution  can  be  readily  established.  The  basic  idea  then,  is  to  annroximate  the  probability 
distribution  of  C  *  by  properly  adjusting  the  distribution  of  C*' ,  so  that  the  expected  value  and  the 
variance  of  the  adjusted  probability  distribution  matches  with  those  of  C*  .  The  approximation  is 
accomplished  by  the  following  steps. 

(/)  Mean  and  variance  of  C*' :  Note  that  C*'  differs  from  the  C  *  in  Case  1  only  by  a  scaling 
factor  R^l-r  ,  more  precisely,  let  C,  *  be  the  C  *  in  Case  1,  then  C*'=  *.  So  The 

expected  value  of  C*'  is  /r'=  R-Jl-rMPP^  (as  in  Case  3),  and  The  variance  of  C*'is 

=  where 

00 

-00 

and  MPPq  is  given  in  equation  (5)  or  (5b).  The  values  of  F  are  given  in  the  first  column  of  the  revised 
Brogden  Table  in  Section  m. 

(/'O  Mean  and  variance  of  C  *  :  From  (8),  by  additivity  of  expected  values,  the  expected  value 
of  C  *  is 

(9)  H  =  'EC*=EK+'EC*'  =  EC*'  =  H'=R^ir^MPP^ 
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because  the  expected  value  of  K  is  0.  By  independence  of  K  and  C*' ,  the  variance  of  C  *  is 
(10)  <T^  =VarA'  +  VarC*'  =  i?V  +  i?^(l-/-)F^ 

(///)  Standardize  C*' :  The  random  variable  C'— — (C*'—ju'')  has  mean  0  and  variance  1  So 

<t' 

the  random  variable  C*  =  +  oC'  has  mean^  and  variance ,  the  same  as  those  of  C  *  .  Hence 

C  *  has  the  probability  distribution  approximately  the  same  that  of  C  * ,  in  the  sense  of  matching  mean 
and  variance.  Approximate  the  distribution  of  C  *  bv  the  distribution  of  C  * .  We  accomplish  this  by 
approximating  the  quantile  function  of  C  *  with  the  quantile  function  of  C  * . 

(/v)  Quantile  function  of  C  * :  From  step  (;),  C*'=  (i?Vl-r)C,  *  and  from  step  (Hi) , 

C*  =  ;/  +  o-C  =  /i  +  cr[(C*’-//') / cr']  =  /r  +  a[(Ry/r^C,  * -//’)  /  o-’] 

*  =  A  +  BC,*, 

where  A  =  la*  and  .S  =  aR^\-r  /  cr' .  By  Proposition  A.5,  the  quantile  function  of  C  *  is 

Q(u)  =  A  +  BQy  (m),  where  Q,  (u)  is  the  quantile  function  of  C,  *  .  Recall  form  step  (i)  that  C,  *  is 

the  C  *  in  Case  1,  whose  quantile  function  is  given  in  equation  (5a)  —  (u)  =  .  Therefor 

substituting  the  values  of  Ay  By  (^)  gives  the  quantile  function  of  C  *  as 

Q(u)  =  A  +  BQ, (M)  =  0  <  M  <  1. 

(v)  Approximate  the  quantile  function  Q(u)  of  C  *  by  that  of  C  * : 

(12)  0<tt<l. 

(v/)  The  adjustment  formula;  The  MPP  is  the  conditional  expected  value  of  C  *  given  it  is 
greater  than  or  equal  to  its  pth  percentile  (i.e.  given  C  Q(r)  )•  So  by  Proposition  A.4, 

1  1 

MPP  =  -^\Q(u)du. 

^  P  p 

Replacing  Q(ti)  by  its  approximation  given  in  equation  (12).  we  have 
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\it 


<7 


^  r 

+ 


a'  J 


By  equation  (7b),  the  integral  in  the  brackets  is  the  MPP^  in  Case  2,  so 

Plugging  in  the  values  of  fj,,  (j,  /J ,  o'  obtained  in  steps  (/)  and  (ii)  gives 


MPP  * 


ylR\l-r)V^  J  I,  y/jl\l-r)V^ 

R^^T^F  Ryfl^  ' 


Ryf 


■MPP,+ 


R^r  +  (l-r)V^ 


V  "  V 

=  Ryfi^AdPP^  +-^Jr  +  (l-r)V^{MPP^  -MPPo). 


MPP. 


The  last  expression  is  exactly  the  adjustment  formula  given  in  Section  m. 


-rMPP^ 
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